Introduction
Peer Models Network is an initiative to make models for healthcare decision-making accessible, transparent and interpretable to all British Columbians. With support from the Canada Foundation for Innovation, BC SUPPORT Unit and BC Academic Health Science Network, the Peer Models Network (PMN) aims to increase engagement between the modelling community and policymakers, interdisciplinary scholars, patients, journalists, and other members of the public.
The PMN centres around a software infrastructure called the Programmable Interface for Statistical and Simulation Models (PRISM). PRISM hosts models on the cloud and provides standard Application Programming Interfaces (APIs) that allows users of different levels of technical backgrounds to interact with models hosted on a cloud. Users can submit queries to cloud-hosted models through a multitude of software platforms, including Microsoft Excel, R, Python, bash scripts, Javascript and custom web apps. This enables direct access to users who lack either the software or hardware infrastructure or the technical expertise to set up models on their own devices. It also enables more technical users to integrate modes with other pieces of software.
PRISM can enhance access to COVID-19 models used to guide public health response to the current pandemic. For example, provincial health officers can use a simple Excel spreadsheet or web app to refit the models themselves and examine results under different assumptions, without facing the technical complexity of running the model. PRISM also empowers the larger research community, journalists, and members of the public to interact with models directly and provide feedback. The Peer Models Network invites public feedback on models, through a private query form and open discussion forum, on the PMN website. Finally, the PMN hosts ‘model companion videos’, educational videos to explain the model’s structure, assumptions, and findings, as well as interviews with modellers and videos on general topics relevant to healthcare decision-making.
Technical Overview
PRISM hosts models on BCNet Educloud servers and provides standard RESP APIs for model functionalities. The user can connect to the model, retrieve a list of its parameters and values, modify them if necessary, submit them to the model on the cloud, and receive results. PRISM also returns binary objects, including graphics that are produced by the model.
In its current implementation, PRISM can hosts models that are developed in R, including those that use Cpp and STAN.
Synchronous APIs calls are currently implemented and support for asynchronous calls that require extra long processing times are under active development.
We have recently deployed the Bayesian SEIR model to estimate physical-distancing effects developed by Anderson et al to our cloud infrastructure. The current version of the R package deployed to the PRISM server is covidseir v0.0.0.9006.
Anderson, S. C., Edwards, A. M., Yerlanov, M., Mulberry, N., Stockdale, J., Iyaniwura, S. A., Falcao, R. C., Otterstatter, M. C., Irvine, M. A., Janjua, N. Z., Coombs, D., & Colijn, C. (2020). Estimating the impact of COVID-19 control measures using a Bayesian model of physical distancing. MedRxiv, 2020.04.17.20070086
Calling COVIDSEIR API through R
We have developed the client R package peermodels that facilitates access to the models hosted on Peer Models Network cloud. The package is currently under active development and can be downloaded from GitHub: Current API Response times Initital handshake : 1-3s Fitting with default parameters : 3-5m Fetching Graphics : 10s
install.packages("remotes", repos="https://mirror.rcg.sfu.ca/mirror/CRAN/")
remotes::install_github("resplab/peermodels")Using the peermodels package, API Key for Dr. Colijn’s Group: YDbxcnNHmf4XoteSmCFHKx the user can authenticate using their personal api key and connect to their model of choice. The Basic work flow for accessing covidseir API from R involves:
Load
peermodelspackageConnect to the model
Fetch default inputs and modify them as needed
Run the model with input parameters and fetch results
Fetch graphics generated by the model
library(peermodels)
connect_to_model("covidseir", api_key = "YDbxcnNHmf4XoteSmCFHKx", bypass_router = T)## Selected model is covidseir## Calling server at http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/json## This is covidseirPrism - PRISM enabled!## $error_code
## [1] 0
##
## $session_id
## [1] ""
##
## $version
## [1] "0.0.8"
##
## $description
## [1] "This is covidseirPrism - PRISM enabled!"If handshake with the model is successful, a response code of 0 is returned with the version of the model on the server. The user can then proceed to get the list of the default input parameters using the get_default_input command:
input <- get_default_input()## Selected model is covidseir## Calling server at http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/jsonnames(input)## [1] "daily_cases" "obs_model" "fit_forecast_days"
## [4] "time_increment" "samp_frac_fixed" "samp_frac_type"
## [7] "samp_frac_seg" "days_back" "R0_prior"
## [10] "phi_prior" "f_prior" "e_prior"
## [13] "samp_frac_prior" "start_decline_prior" "end_decline_prior"
## [16] "f_ramp_rate" "rw_sigma" "seed"
## [19] "chains" "fit_iter" "N_pop"
## [22] "pars" "i0_prior" "state_0"
## [25] "save_state_predictions" "delay_scale" "delay_shape"
## [28] "ode_control" "project_forecast_days" "f_fixed_start"
## [31] "f_fixed" "forecast_iter" "return_states"
## [34] "imported_cases" "imported_window"input$daily_cases## [1] 0 0 1 3 1 8 0 6 5 0 7 7 18 9 22 38 53 45 40 77 76 48 67 78 42
## [26] 66 67 92 16 70 43 53 55 53 29 26 37 25 45 34 40 35The function model_run can be used to run the model with default inputs. If the model generates graphical output, they can be retrieved using the draw_plots() function:
results <- model_run(input)
names(results)## [1] "model" "projected_day" "projected_data_type"
## [4] "projected_mu" "projected_y_rep"draw_plots()
Changing inputs
The code snippet below, automatically downloads latest COVID-19 case numbers for British Columbia, and reruns the simulation using different assumptions.
library(stringr)
library(dplyr)
library(tidyr)
library(readr)
library(lubridate)
library(peermodels)
url <- "https://docs.google.com/spreadsheets/d/1ad7-09_Jn6AxsdkVPE33T-iLfGpPRmd3piXQqFiVeas/export?&format=csv"
CanadaCases <- read_csv(url)## Parsed with column specification:
## cols(
## .default = col_double(),
## prname = col_character(),
## prnameFR = col_character(),
## date = col_character(),
## percentrecover = col_character()
## )## See spec(...) for full column specifications.covidCases <- CanadaCases %>% rename (name = "prname") %>% rename (Cases = "numconf") %>% mutate(date=dmy(date)) %>%
filter (name!="Canada") %>% filter (date!=today())
bcCases <- covidCases %>% filter (name == "British Columbia")
connect_to_model("covidseir", api_key = "YDbxcnNHmf4XoteSmCFHKx", bypass_router = T)## Selected model is covidseir## Calling server at http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/json## This is covidseirPrism - PRISM enabled!## $error_code
## [1] 0
##
## $session_id
## [1] ""
##
## $version
## [1] "0.0.8"
##
## $description
## [1] "This is covidseirPrism - PRISM enabled!"input <- get_default_input()## Selected model is covidseir## Calling server at http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/jsonfirst <- length(bcCases$numtoday)-42+1
last <- length(bcCases$numtoday)
input$daily_cases <- bcCases$numtoday[first:last]
# Example assumed sampling fractions of positive cases:
s1 <- c(rep(0.1, 13), rep(0.2, length(input$daily_cases) - 13))
samp_frac_seg <- c(rep(1, 13), rep(2, length(input$daily_cases) - 13))
s2 <- rep(0.07, length(input$daily_cases)) # Assuming 7\% of positive individuals are hospitalized
input$samp_frac_fixed <- rep(0.1, length(input$daily_cases))
input$fit_iter <- 100
input$chains <- 1
input$f_fixed <- rep(0.1, 90)
results <- model_run(input)## Selected model is covidseir
## Calling server at http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/jsondraw_plots()## Session ID:x00abee09e0cbde## Calling server at http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/tmp/x00abee09e0cbde/
Calling COVIDSEIR API through Python
COVIDSEIR on the PRISM cloud can also be called through Python, as shown below:
import json
import requests
api_key = 'YDbxcnNHmf4XoteSmCFHKx'
url = 'http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/json'
headers = {'x-prism-auth-user': 'YDbxcnNHmf4XoteSmCFHKx'}
default_input = requests.post(url, headers=headers,json = {"func":["get_default_input"]})
# json_default_input = json.loads(default_input.json()[0]) # OR:
json_default_input = json.loads(json.loads(default_input.text)[0])
response = requests.post(url, headers=headers,json = {"func":["prism_model_run"], "model_input":json_default_input})
results = json.loads(response.text)Calling COVIDSEIR API through Shell
Models on PRISM can be invoked through standard curl calls. The following code snippet illustrates the approprieate structure of the call in Ubuntu:
curl \
-X POST \
-H "x-prism-auth-user: YDbxcnNHmf4XoteSmCFHKx" \
-H "Content-Type: application/json" \
-d '{"func":["prism_model_run"],"model_input":{"daily_cases":[0,0,1,3,1,8,0,6,5,0,7,7,18,9,22,38,53,45,40,77,76,48,67,78,42,66,67,92,16,70,43,53,55,53,29,26,37,25,45,34,40,35],"obs_model":["NB2","Poisson"],"fit_forecast_days":[0],"time_increment":[0.25],"samp_frac_fixed":[0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"samp_frac_type":["fixed","estimated","rw","segmented"],"samp_frac_seg":{},"days_back":[45],"R0_prior":[0.9555,0.2],"phi_prior":[1],"f_prior":[0.4,0.2],"e_prior":[0.8,0.05],"samp_frac_prior":[0.4,0.2],"start_decline_prior":[2.7081,0.05],"end_decline_prior":[3.091,0.05],"f_ramp_rate":[0],"rw_sigma":[0.1],"seed":[42],"chains":[1],"fit_iter":[100],"N_pop":[5100000],"pars":[5,0.2,1,0.05,0.1,0.02,1],"i0_prior":[2.0794,1],"state_0":[0.4,0.1,0.5,0,0,0.4,0.1,0.5,0,0],"save_state_predictions":[false],"delay_scale":[9.85],"delay_shape":[1.73],"ode_control":[1e-07,1e-06,1000000],"project_forecast_days":[100],"f_fixed_start":[53],"f_fixed":[0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.7,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2,0.2],"forecast_iter":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],"return_states":[false],"imported_cases":[0],"imported_window":[1]}} ' \
http://model-covidseir.cp.prism-ubc.linaralabs.com/ocpu/library/covidseirPrism/R/gateway/jsonIntergration with Shiny, Excel, Javascript, etc.
For users of the model who do not have a coding background, models hosted on PRISM through web apps, or simple spreadsheets that allows them to change inputs and re-run the model on the cloud. For an example of a PRISM-enabled Excel sheet, see the Excel spreadsheet for the Evaluation Platform in COPD (EPIC) model. Note that the Excel sheets work on Microsoft Windows only.
PRISM models can also be called through Javascript and practically any programming language that supports modern web APIs.
Known Issues
The current experimental implementation of the covidseir package has some limitations. All inputs can be currently be modified except for pars, and state_0. Additionally, depending on the number of chains and iterations, the html connection might be dropped for calls that more than a dozen minutes. We are actively working to implement asynchronous support which would email the results back to the user when multi-hour model runs are requested.